Objectives

  • Feedback on the slides/presentation
  • Bayesian/Frequentist stopping boundary
  • Explanation of differences using different priors
  • Next steps

APTS Durham

APTS Durham

  • Use version control (Git)

APTS Durham

  • Attend lots of seminars
  • Use version control (Git)

APTS Durham

  • Read the latest and first paper on a topic
  • Attend lots of seminars
  • Use version control (Git)

APTS Durham

  • Get the simplest case working
  • Read the latest and first paper on a topic
  • Attend lots of seminars
  • Use version control (Git)

APTS Durham

  • Get the simplest case working Otherwise, go back to the drawing board
  • Read the latest and first paper on a topic
  • Attend lots of seminars
  • Use version control (Git)

Presentation of results

  • Change from plots of \(\theta\) to plots of ratios.
  • Include plots which overlay frequentist and priors using pooled with facet for each analysis.
    • Do the same for the group based approach

Stopping Boundary

Stopping Boundary

Stopping Boundary

  • Bayesian stopping rule

Stopping Boundary

  • Bayesian stopping rule
  • Bayesian allocation ratio only

Stopping Boundary

  • Bayesian stopping rule
  • Bayesian allocation ratio only

Stopping Boundary

Irving John Good (1916 - 2009)

The subjectivist (i.e. Bayesian) states his judgements, whereas the objectivist sweeps them under the carpet by calling assumptions knowledge, and he basks in the glorious objectivity of science.

Stopping Boundary

Irving John Good (1916 - 2009)

The subjectivist (i.e. Bayesian) states his judgements, whereas the objectivist sweeps them under the carpet by calling assumptions knowledge, and he basks in the glorious objectivity of science.

  • What assumptions do we want to make?
  • Interpretability
  • Option 3
  • Time constraint

Prior on individual treatments vs treatment effect directly

Maximum likelihood estimate

Maximum likelihood estimate

Generate data from group j using independent normal distributions with mean \(\mu_j\) and a known common variance \(\sigma^2\).

\[ X_{i,j} \sim N(\mu_j, \sigma^2) \]

where \(j=1,2\) and \(i=1,\ldots,n_j\).

Maximum likelihood estimate

Generate data from group j using independent normal distributions with mean \(\mu_j\) and a known common variance \(\sigma^2\).

\[ X_{i,j} \sim N(\mu_j, \sigma^2) \]

where \(j=1,2\) and \(i=1,\ldots,n_j\).

Using pooled data over groups, we get the following expression for our estimate of the treatment effect \(\theta\) at analysis \(k\)

\[ \hat{\theta}_{p,K} = \frac{\sum_{k=1}^{K}\bar{X}_1^k n_1^k}{\sum_{k=1}^{K}n_1^k} - \frac{\sum_{k=1}^{K}\bar{X}_2^k n_2^k}{\sum_{k=1}^{K}n_2^k} \]

Maximum likelihood estimate

simulate_RAR <- function(N, K, mu_1, mu_2, sigma, I_theta_fix,delta) {
  # Ratio, estimates of mu, theta
  muhat_1 <- muhat_2 <- ratio <- numeric(K)
  theta_hat <- rep(NA, K)
  # data from trial
  x_1 <- x_2 <- numeric(0)
  # Information
  I1 <- I2 <- numeric(K)
  # Number of patients per arm
  n1 <- n2 <- n1.new <- n2.new <- w <- numeric(K)
  S1 <- N1 <- S2 <- N2 <- 0
  for (k in 1:K) {
    if (k == 1) {
      # Initial equal allocation
      ratio[k] <- 1
      # Enroll 20 patients when N = 100, K=5
      n1.new[k] <- n2.new[k] <- N / K
      n1[k] <- n2[k] <- N / K
    } else {
      # Define ratio based on formula with previous estimate of theta
      ratio[k] <-  a ** (theta_hat[k - 1]/ (2 * delta))
      # Patients in N1
      n1[k] <- (k / 5) * (sigma ^ 2) * I_theta_fix * (1 + ratio[k])
      # Patient in N2
      n2[k] <- (k / 5) * (sigma ^ 2) * I_theta_fix * (1 + (1 / ratio[k]))
      # Difference in patients from previous
      n1.new[k] <- n1[k] - n1[k - 1]
      n2.new[k] <- n2[k] - n2[k - 1]
      if(n1.new[k]<=0){
        n1[k]= n1[k - 1]+0.00001
        n1.new[k] = n1[k] - n1[k - 1]
        #n2[k] = (((k/5)*I_theta_fix)^(-1)-1/(n2/sigma^2))^(-1)
        n2[k] = (k*n1[k]*sigma^2*I_theta_fix)/(5*n1[k]-k*sigma^2*I_theta_fix)
        n2.new[k] = n2[k]-n2[k-1]
      }
      if(n2.new[k]<=0){
        n2[k]= n2[k - 1]+0.00001
        n2.new[k] = n2[k] - n2[k - 1]
        #n1[k] = (((k/5)*I_theta_fix)^(-1)-1/(n2[k]/sigma^2))^(-1)
        n1[k] = (k*n2[k]*sigma^2*I_theta_fix)/(5*n2[k]-k*sigma^2*I_theta_fix)
        n1.new[k] = n1[k]-n1[k-1]
      }
    }
    #Sample once from normal distribution
    x_1[k] <- rnorm(1, mu_1, sqrt(sigma^2/n1.new[k]))
    x_2[k] <- rnorm(1, mu_2, sqrt(sigma^2/n2.new[k]))
    
    muhat_1[k] <- sum(n1.new[1:k] * x_1[1:k]) / sum(n1.new)
    muhat_2[k] <- sum(n2.new[1:k] * x_2[1:k]) / sum(n2.new)
    theta_hat[k] <- muhat_1[k] - muhat_2[k]
  }
  return(list(n1 = n1, n2 = n2, theta_hat = theta_hat))
}

Maximum likelihood estimate

Designs with no early stopping
𝜃
Target
RAR Using Pooled θ
RAR Using Grouped θ
EN1 EN2 total EN1 EN2 total EN1 EN2 total
-½𝛿 85.4 120.7 206.1 86.4 (0) 122.8 (0) 209.2 90 (0) 120.1 (0) 210.1
0𝛿 100 100 200 101.4 (0) 101.5 (0) 202.9 102.5 (0) 102.5 (0) 205
½𝛿 120.7 85.4 206.1 122.8 (0) 86.4 (0) 209.2 120.1 (0) 90 (0) 210.1
1𝛿 150 75 225 153 (0) 75.7 (0) 228.7 145 (0) 81.2 (0) 226.2
3/2𝛿 191.4 67.7 259.1 195.8 (0) 68.2 (0) 263.9 180.1 (0) 75 (0) 255.1
2𝛿 250 62.5 312.5 256.4 (0.1) 62.8 (0) 319.3 229.9 (0.1) 70.6 (0) 300.5
Results are based on 1e+06 simulations; mean (standard error) to 1 decimal place

Prior on individual \(\mu\)’s

Prior on individual \(\mu\)’s

Prior on individual \(\mu\)’s

Suppose the responses of patients from treatment group j are iid normals

\[ X_{i,j}|\mu_j,\tau \sim N(\mu_j, \tau^{-1}) \]

Prior on individual \(\mu\)’s

Suppose the responses of patients from treatment group j are iid normals

\[ X_{i,j}|\mu_j,\tau \sim N(\mu_j, \tau^{-1}) \]

With priors on \(\mu\) for each treatment j

\[ \mu_j \sim N(\mu_{0,j}, \tau_{0,j}^{-1}) \]

Prior on individual \(\mu\)’s

Suppose the responses of patients from treatment group j are iid normals

\[ X_{i,j}|\mu_j,\tau \sim N(\mu_j, \tau^{-1}) \]

With priors on \(\mu\) for each treatment j

\[ \mu_j \sim N(\mu_{0,j}, \tau_{0,j}^{-1}) \]

Then the posterior \(\theta|\bar{x}_1^k,\bar{x}_2^k\) has mean

\[ \frac{\tau_{0,1}\mu_{0,1}+\sum_{i=1}^{k} n_1^i \tau \mu_1^k}{\tau_{0,1}+\sum_{i=1}^k n_1^i \tau} - \frac{\tau_{0,2}\mu_{0,2}+\sum_{i=1}^{k} n_2^i \tau \mu_2^k}{\tau_{0,2}+\sum_{i=1}^k n_2^i \tau} \]

Prior on individual \(\mu\)’s

Suppose the responses of patients from treatment group j are iid normals

\[ X_{i,j}|\mu_j,\tau \sim N(\mu_j, \tau^{-1}) \]

With priors on \(\mu\) for each treatment j

\[ \mu_j \sim N(\mu_{0,j}, \tau_{0,j}^{-1}) \]

Then the posterior \(\theta|\bar{x}_1^k,\bar{x}_2^k\) has mean

\[ \frac{\tau_{0,1}\mu_{0,1}+\sum_{i=1}^{k} n_1^i \tau \mu_1^k}{\tau_{0,1}+\sum_{i=1}^k n_1^i \tau} - \frac{\tau_{0,2}\mu_{0,2}+\sum_{i=1}^{k} n_2^i \tau \mu_2^k}{\tau_{0,2}+\sum_{i=1}^k n_2^i \tau} \]

and variance

\[ 1/(\tau_{0,1}+\sum_{i=1}^k n_1^i \tau) + 1/(\tau_{0,2}+\sum_{i=1}^k n_2^i \tau) \]

Prior on individual \(\mu\)’s

Suppose the responses of patients from treatment group j are iid normals

\[ X_{i,j}|\mu_j,\tau \sim N(\mu_j, \tau^{-1}) \]

With priors on \(\mu\) for each treatment j

\[ \mu_j \sim N(\mu_{0,j}, \tau_{0,j}^{-1}) \]

Then the posterior \(\theta|\bar{x}_1^k,\bar{x}_2^k\) has mean

\[ \frac{\color{gray}{\tau_{0,1}\mu_{0,1}+}\sum_{i=1}^{k} n_1^i \tau \mu_1^k}{\color{gray}{\tau_{0,1}}+\sum_{i=1}^k n_1^i \tau} - \frac{\color{gray}{\tau_{0,2}\mu_{0,2}+}\sum_{i=1}^{k} n_2^i \tau \mu_2^k}{\color{gray}{\tau_{0,2}}+\sum_{i=1}^k n_2^i \tau} \]

and variance

\[ 1/(\tau_{0,1}+\sum_{i=1}^k n_1^i \tau) + 1/(\tau_{0,2}+\sum_{i=1}^k n_2^i \tau) \]

Prior on individual \(\mu\)’s

posterior_mean <- function(prior_mean, prior_tau, data, tau, n,k) {
  result <- tryCatch({
    # Check if the lengths of n and data are compatible
    if (length(n) != length(data)) {
      stop("Length of n and data must be the same. Found n of length ", length(n), " and data of length ", length(data))
    }
    # Calculate post mean
    posterior_mean_result <- ((prior_tau) * prior_mean + sum(n * (tau) * (data))) / (sum(n* tau) + prior_tau)
    return(posterior_mean_result)
    
  }, error = function(e) {
    # Catch the error and return a message
    cat("Error in posterior_mean (K=):",k, e$message, "\n")
    cat("n: ",paste(n),"\n")
    cat("data: ",paste(data),"\n")
    return(NA)  # Return NA or any default value in case of error
  })
  
  return(result)
}

x_1[k] <- rnorm(1, mean = mu_1, sd = sqrt(sigma ^ 2 / n1.new[k]))
x_2[k] <- rnorm(1, mean = mu_2, sd =  sqrt(sigma ^ 2 / n2.new[k]))

# Posterior variances
tau_n1[k] <- posterior_tau(prior_tau = tau_01,
                           tau = tau,
                           n = n1.new[1:k])

tau_n2[k] <- posterior_tau(prior_tau = tau_02,
                           tau = tau,
                           n = n2.new[1:k])
# Posterior means
mu_n1[k] <- posterior_mean(
  prior_mean = mu_01,
  n = n1.new[1:k],
  tau = tau,
  prior_tau = tau_01,
  data = x_1[1:k],
  k = k
)

mu_n2[k] <- posterior_mean(
  prior_mean = mu_02,
  n = n2.new[1:k],
  tau = tau,
  prior_tau = tau_02,
  data = x_2[1:k],
  k = k
)

theta_hat[k] <- mu_n1[k] - mu_n2[k]

Prior on individual \(\mu\)’s

No early stopping using frequentist and bayesian methods with pooled θ
EN1_pool EN2_pool total_pool EN1_brar_1 EN2_brar_1 total_brar_1 EN1_brar_2 EN2_brar_2 total_brar_2 EN1_brar_3 EN2_brar_3 total_brar_3 EN1_brar_4 EN2_brar_4 total_brar_4
-1/2𝛿 86.4 (0) 122.8 (0) 209.2 (0) 86.4 (0) Equals 122.8 (0) Equals 209.2 (0) Equals 94.5 (0) Arrow Up 107.9 (0) Arrow Down 202.5 (0) Arrow Down 111 (0) Arrow Up 91.3 (0) Arrow Down 202.4 (0) Arrow Down 83.3 (0) Arrow Down 125 (0) Arrow Up 208.3 (0) Arrow Down
0𝛿 101.4 (0) 101.5 (0) 202.9 (0) 101.5 (0) Equals 101.4 (0) Equals 202.9 (0) Equals 107.8 (0) Arrow Up 94.7 (0) Arrow Down 202.5 (0) Arrow Down 121.1 (0) Arrow Up 85.5 (0) Arrow Down 206.6 (0) Arrow Up 83.3 (0) Arrow Down 125 (0) Arrow Up 208.3 (0) Arrow Up
1/2𝛿 122.8 (0) 86.4 (0) 209.2 (0) 122.8 (0) Equals 86.4 (0) Equals 209.2 (0) Equals 126.1 (0) Arrow Up 84 (0) Arrow Down 210.1 (0) Arrow Up 133.8 (0) Arrow Up 80.1 (0) Arrow Down 214 (0) Arrow Up 83.4 (0) Arrow Down 125 (0) Arrow Up 208.3 (0) Arrow Down
1𝛿 153 (0) 75.7 (0) 228.7 (0) 153.1 (0) Equals 75.7 (0) Equals 228.7 (0) Equals 151.5 (0) Arrow Down 75.5 (0) Arrow Down 227 (0) Arrow Down 150.4 (0) Arrow Down 75.2 (0) Arrow Down 225.6 (0) Arrow Down 83.4 (0) Arrow Down 124.9 (0) Arrow Up 208.3 (0) Arrow Down
3/2𝛿 195.8 (0) 68.2 (0) 263.9 (0) 195.8 (0) Equals 68.2 (0) Equals 264 (0) Equals 187.7 (0) Arrow Down 68.8 (0) Arrow Up 256.5 (0) Arrow Down 172.6 (0) Arrow Down 70.6 (0) Arrow Up 243.2 (0) Arrow Down 83.4 (0) Arrow Down 124.9 (0) Arrow Up 208.3 (0) Arrow Down
2𝛿 256.4 (0.1) 62.8 (0) 319.3 (0.1) 256.4 (0.1) Equals 62.8 (0) Equals 319.2 (0.1) Equals 239.6 (0) Arrow Down 63.7 (0) Arrow Up 303.2 (0) Arrow Down 203.1 (0) Arrow Down 66.6 (0) Arrow Up 269.6 (0) Arrow Down 83.4 (0) Arrow Down 124.9 (0) Arrow Up 208.3 (0) Arrow Down
Based on1e+06Simulations; mean (standard error) to 1 decimal place
Prior 1:mu_1: 0𝛿, mu_2: 0, tau_01: 1e-05, tau_02: 1e-05, theta_tau0: 5e-06, theta: 0𝛿 Prior 2:mu_1: 1𝛿, mu_2: 0, tau_01: 20, tau_02: 20, theta_tau0: 10, theta: 1𝛿 Prior 3:mu_1: 1𝛿, mu_2: 0, tau_01: 100, tau_02: 100, theta_tau0: 50, theta: 1𝛿 Prior 4:mu_1: -0.584962500721156𝛿, mu_2: 0, tau_01: 100000, tau_02: 100000, theta_tau0: 50000, theta: -0.584962500721156𝛿

Priors on \(\theta\)

Now suppose there are priors directly on \(\theta\)

\[ \theta \sim N(\theta_0,\tau_{0,\theta}) \]

Priors on \(\theta\)

Now suppose there are priors directly on \(\theta\)

\[ \theta \sim N(\theta_0,\tau_{0,\theta}) \] And generate data,

\[ Y^k = \bar{X}_1^k - \bar{X}_2^k \sim N(\theta,\tau_y^{-1}) \]

Where

\[ \tau_y^k = (\sigma^2/n_1^k + \sigma^2/n_2^k)^{-1} \]

Priors on \(\theta\)

Then the posterior distribution \(\theta|\bar{y}^k\) has mean

\[ \frac{\theta_0\tau_{0,\theta}+\sum_{i=1}^k y^i\tau_y^i}{\tau_{0,\theta}+\sum_{i=1}^k \tau_y^i} \]

and variance

\[ 1/(\tau_{0,\theta}+\sum_{i=1}^k \tau_y^i) \]

Where

\[ \tau_y = (\sigma^2/n_1^k + \sigma^2/n_2^k)^{-1} \]

Priors on \(\theta\)

No early stopping using frequentist and bayesian methods with pooled θ
EN1_pool EN2_pool total_pool EN1_brar_1 EN2_brar_1 total_brar_1 EN1_brar_2 EN2_brar_2 total_brar_2 EN1_brar_3 EN2_brar_3 total_brar_3 EN1_brar_4 EN2_brar_4 total_brar_4
-1/2𝛿 86.4 (0) 122.8 (0) 209.2 (0) 86.4 (0) Equals 123.5 (0) Arrow Up 209.9 (0) Arrow Up 94.8 (0) Arrow Up 108 (0) Arrow Down 202.8 (0) Arrow Down 114.1 (0) Arrow Up 89.4 (0) Arrow Down 203.6 (0) Arrow Down 125 (0) Arrow Up 83.4 (0) Arrow Down 208.3 (0) Arrow Down
0𝛿 101.4 (0) 101.5 (0) 202.9 (0) 101.7 (0) Arrow Up 101.7 (0) Arrow Up 203.4 (0) Arrow Up 109 (0) Arrow Up 94 (0) Arrow Down 203 (0) Arrow Up 124.5 (0) Arrow Up 83.9 (0) Arrow Down 208.4 (0) Arrow Up 125 (0) Arrow Up 83.3 (0) Arrow Down 208.3 (0) Arrow Up
1/2𝛿 122.8 (0) 86.4 (0) 209.2 (0) 123.5 (0) Arrow Up 86.4 (0) Equals 209.9 (0) Arrow Up 127.6 (0) Arrow Up 83.5 (0) Arrow Down 211.1 (0) Arrow Up 136.6 (0) Arrow Up 79.2 (0) Arrow Down 215.8 (0) Arrow Up 125 (0) Arrow Up 83.3 (0) Arrow Down 208.3 (0) Arrow Down
1𝛿 153 (0) 75.7 (0) 228.7 (0) 154.6 (0) Arrow Up 75.6 (0) Arrow Down 230.2 (0) Arrow Up 152.1 (0) Arrow Down 75.4 (0) Arrow Down 227.6 (0) Arrow Down 150.6 (0) Arrow Down 75.1 (0) Arrow Down 225.7 (0) Arrow Down 125 (0) Arrow Down 83.3 (0) Arrow Up 208.3 (0) Arrow Down
3/2𝛿 195.8 (0) 68.2 (0) 263.9 (0) 198.5 (0) Arrow Up 68.1 (0) Arrow Down 266.5 (0) Arrow Up 184.1 (0) Arrow Down 69.4 (0) Arrow Up 253.4 (0) Arrow Down 166.7 (0) Arrow Down 71.7 (0) Arrow Up 238.3 (0) Arrow Down 125 (0) Arrow Down 83.3 (0) Arrow Up 208.4 (0) Arrow Down
2𝛿 256.4 (0.1) 62.8 (0) 319.3 (0.1) 260.7 (0.1) Arrow Up 62.8 (0) Arrow Down 323.5 (0.1) Arrow Up 225.6 (0) Arrow Down 64.8 (0) Arrow Up 290.4 (0) Arrow Down 185.2 (0) Arrow Down 68.7 (0) Arrow Up 253.9 (0) Arrow Down 125.1 (0) Arrow Down 83.3 (0) Arrow Up 208.4 (0) Arrow Down
Based on1e+06Simulations; mean (standard error) to 1 decimal place
Prior 1:mu_1: 0𝛿, mu_2: 0, tau_01: 1e-06, tau_02: 1e-06, theta_tau0: 5e-07, theta_0: 0, theta: 0𝛿 Prior 2:mu_1: 1𝛿, mu_2: 0, tau_01: 20, tau_02: 20, theta_tau0: 10, theta_0: 0.4584195253573, theta: 1𝛿 Prior 3:mu_1: 1𝛿, mu_2: 0, tau_01: 100, tau_02: 100, theta_tau0: 50, theta_0: 0.4584195253573, theta: 1𝛿 Prior 4:mu_1: 0.584962500721156𝛿, mu_2: 0, tau_01: 1e+05, tau_02: 1e+05, theta_tau0: 50000, theta_0: 0.268158231932412, theta: 0.584962500721156𝛿

Differences in using direct vs individual priors

  • Individual priors on \(\mu\)’s

Mean

\[ \frac{\tau_{0,1}\mu_{0,1}+\sum_{i=1}^{k} n_1^i \tau \mu_1^k}{\tau_{0,1}+\sum_{i=1}^k n_1^i \tau} - \frac{\tau_{0,2}\mu_{0,2}+\sum_{i=1}^{k} n_2^i \tau \mu_2^k}{\tau_{0,2}+\sum_{i=1}^k n_2^i \tau} \] Variance

\[ 1/(\tau_{0,1}+\sum_{i=1}^k n_1^i \tau) + 1/(\tau_{0,2}+\sum_{i=1}^k n_2^i \tau) \]

  • Direct priors on \(\theta\)

Mean

\[ \frac{\theta_0\tau_{0,\theta}+\sum_{i=1}^k y^i\tau_y^i}{\tau_{0,\theta}+\sum_{i=1}^k \tau_y^i} \] Variance

\[ 1/(\tau_{0,\theta}+\sum_{i=1}^k \tau_y^i) \]

Where

\[ \tau_y = (\sigma^2/n_1^k + \sigma^2/n_2^k)^{-1} \]

Differences in using direct vs individual priors

The posterior variance of \(\theta\) when separate prior distributions are given is always smaller than that when only the prior distribution for \(\theta\) is used.1

Differences in using direct vs individual priors

  • Assess difference using idea of coupling
  • Primarily interested in the vague prior case

Differences in using direct vs individual priors

Distribution of theta_hat at each analysis K using individual priors for treatments and a prior for theta directly using coupling

Differences in using direct vs individual priors

Means and variances of theta_hat at each analysis K using individual priors for treatments and a prior for theta directly using coupling

Conferences

Three conference deadlines in May:

  • Samba Bath 2nd May 2025
  • RSS Rapid-fire talks Edinburgh 9th May 2025
  • Bayes Pharma Netherlands 15th May
Stallard, Nigel, Susan Todd, Elizabeth G. Ryan, and Simon Gates. 2020. “Comparison of Bayesian and Frequentist Group-Sequential Clinical Trial Designs.” BMC Medical Research Methodology 20 (1): 4. https://doi.org/10.1186/s12874-019-0892-8.